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ABSTRACT 

This report presents findings of an evaluation, 
conducted in 1994, of 10 Leadership in Accountability Demonstration 
(LAD) schools in the San Diego City Schools. The report provides 
information on the status of accountability implementation, 
particularly regarding the schools' recognition and intervention 
practices. The evaluation sought to determine the extent to vrhich the 
LAD schools articulated and impleaented an accountability system and 
addressed each of the four elements of accountability. The four 
elements of accountability included standards, related assessments, 
recognition and intervention measures, and public reporting 
practices. Data were obtained from school self-assessments and onsite 
observation (validation review). Findings show that the average 
highest ratings were in the area of standards, where most schools 
placed at performance levels 3 and A (on a scale of 1 to A). The next 
highest ratings were for assessment, followed by articulation and 
implementation of the accountability system. Slightly lower average 
ratings were found for public reporting and recognition/intervention. 
The 10 LAD schools have continued to move forward in their 
accountability efforts. In the first project year, sites focused on 
the identification of standards and related learner outcomes and 
began to investigate performance-based assessments to measure 
progress toward those outcomes. During the second year, LAD schools 
began using their work in standards and assessment to identify 
appropriate recognition and intervention strategies and report on 
student achievement and programs toward meeting school goals and 
expectations. Five recommendation* to improve the LAD system include: 
(1) promote the expansion of the accountability self-assessment 
process; (2) develop and rtisseninate suggested strategies to help 
sites make the "n>xt step" in progressing along the accountability 
continuums; (3) investigate ways to continue building assessment 
capacity at the site and district level; (A) provide support and 
training for schools in the identification and application of 
appropriate means of recognizing success and the modification of 
programs and practices to intervene when progress is not made; and 
(5) bring into alignment all accountability systems and requirements 
from national initiatives (Goals 2000, Title I, School-to-Work) , 
state projects (Program Quality Review, Western Association of 
Schools and Colleges Accreditation, state assessments), and district 
activities (LAD, Comprehensive Site Plan, professional development, 
district assessment). One table and five figures are included. 
Appendices contain an accountability process overview, the 1994-95 
LAD evaluation plsn, accountability rubrics, accountability rubic 
summary matrix, and a self-assessment and validation review rating 
sheet. (LMI) 
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EXECUTIVE SUMMARY 

ISSUWCONCBRN 

In June 1994, the board reviewed p, Progress Report on the Leadership in Accomtability 
Demonstration (LAD) Schools. Key findings from the study indicated that, apppoxiraately 
six months into the accountability implementation process, LAD schools were progressing in 
then* work in tlie areas of standards and assessment but were only in the initial stages of 
addressing the elements of recognition/intervention and public reporting. After discuSwSion of 
the progress study findings, the board requested additional study of the ten LAD schools as 
well as an interim report by January 1995 on the LAD school accountability efforts. This 
report presents findings from the first part of a year-long evaluation study of the original ten 
LAD schools. The interim study involved a school self-assessment and validation review 
process to examine the schools' status in each of the four elements of accountability as well 
as the overall articulation and implementation of an accoimtability system. 

BACKGROUND 

The Report and Recommendations of the Student Achievement Accountability Committee 
(SAAC), approved by the board in March 1993, identified four major constituent elements of 
an accountability system-standards, related assessments, recognition and intervention 
measures, and public reporting practices. The report also delineated a three-phase 
accountability implementation process. Ten Leadership in Accountability Demonstration 
(LAD) schools participated in training and initial planning during the Peve^opnipnt and 
Assessment Phase in the spring of 1993. They are now in the Resource/ Assistance and 
Restructuring Phase which was designed to take place during the 1993-94 and 1994-95 
school years. Progress was to be assessed at the end of those two years using multiple 
indicators, followed by a Renewal Phase involving the implementation of recognition and 
intervention measures based on each school's progress in demonstrating accountability. 

In February and March of 1994, a progress study was conducted to examine v/hat the LAD 
schools had accomplished during the first half of the 1993-94 school year to implement the 
four elements of accountability. Among the key findings from this study was the indication 
that LAD schools had dedicated considerable time to the identification of standards and 
learner outcomes, and to the selection and/or development of related assessments. However, 
the schools were still in the begijining stages of addressing the accountability elements of 
recognitionAmtervention and public reporting. This was to be expected, since recognizing, 
intervening, and reporting need to be based on student achievement inJForraation from site 
assessments which identify progress toward the standards and learner outcomes. 

During board discussion of the progress report, it was requested that interim information be 
collected on the status of accountability implementation, e;^pcciaily regarding recognition and 
intervention, and reported at a work session in January 1995. Therefore, the first part of a 
year-long study occurred during the fall of 1994 and involved site self-assessments of the 
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LAD schools' current status in the implementation of the four elements of accountability. 
These self-assessment results were tlien validated by central office reviewers. The complete 
evaluarion is proposed to include formal data collection at the end of tlie 1994-95 school year 
through interviews, surveys, observations, and document analysis, with the results presented 
to the board in September 1995. In addition, the District Achievement Accountability 
Committee (DAAC) will gather data during the 1994-95 school year to review each school's 
progress in demonstrating accountability, and will make reconmiendations to the board in the 
fall of 1995 regarding recognition and intervention measures. 

PURPOSE AND METHODOI/XIY 

The purpose of the first part of the evaluation study was to address the following questions: 

..• "ps* ^hat extent have the LAD schools articulated and implemented an accountability system 
Bpd aBdiessed each of the four elements of accountability? 

• How do the schools rate their own achievement in each of these areas? 

• Are school self-assessments validated by external reviewers? 

In the fall of 1994, four-point holistic evaluation rubrics were designed for each of tlie 
accountability elements, using the guiding principles and implementation guidelines from the 
S AAC report along with expertise from teachers and odier staff members at the LAD schools. 
A fifth rubric was added to address the overall articulation and implementation of an 
accountability system. During November, each LAD school conducted a self-assessment 
using the five rubrics; they identified their performance level for each rubric, the types of 
evidence reviewed, and comments or observations. In early December, staff members from 
the Planning and Accountability Team (PAT), accompanied by one or two observers from the 
DAAC, visited each school to review site evidence and either validate the school rating or 
recommend a different performance level. 

SITMMARY OF KEY FINDTNGS 

Self-Assessment Process and Validation Review Visits 

School self-assessments were conducted imder the leadership of site LAD teams or 
accountability committees. At about half the schools this process involved teachers or the 
entire staff while at other sites ratings were completed by the LAD team. To conduct the 
validation reviews, a PAT reviewer met with site administrators or members of the site 
leadership team at each school. Some schools provided the validation reviewer with 
extensive docimientation to support the self-assessment ratings while others gave primarily 
verbal testimony. There was agreement about 86 pei^cent of the time between the validation 
review and the self-assessment ratings. 

Accountability Performangg LoveL? 

The highest average ratings were in the area of S.taiiil£rdfi. witli most schools placed at 
Performance Levels 3 and 4. The next highest ratings were for Assessment and then 
Articulation and Implementation of an Accountability System. Slightly lower average ratings 
were found for Public Reporting and then Recognition/ Intervention : in these areas most 
schools were at Performance Levels 2 and 3. No school was rated at Level 1 on any rubric, 
while validation reviewers assigned four ratings at Level 4 (one in standards, one in 
assessment, and two in public reporting). 

Articulation and Implementation of an Accountabilitv System . Validation reviewers placed 
nine of the ten schools at Performance Level 3 on this rubric. The reviews noted that schools 
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seem committed to implementing accountability, and are movijig forward in addressing aJl 
accountability elements. Schools with new Comprehensive Site Plans indicated that the 
Annual Action Plan has assisted their processes for articulation and implementation of 
accountability. Weaknesses in this aiea were primarily related to stakeholder involvement. 
While schools and reviewers reported that teachers were highly involved and knowledgeable 
about the accountability system, there have been varying degrees of articulation to and 
involvement of other staff, parents, and students. 

gtandardg . Validatiwi review results showed one school rated at Performance Level 4 and 
nine schools at Level 3 in this area. All schools have identified standards and developed 
related learner outcomes or grade level expectancies based on Uie state frameworks, district 
goals and expectations, draft content standards, and other resources. So far the focus has 
been in language arts, althougli some sites have begun addressing mathematics as well. At 
most schools, the learner outcomes have been developed collaboratively by teachers with at 
least some involvement of other stakeholder groups. Staff turnover and high mobility have 
required a commitment at some schools to ongoing articulation in this area. 

Assessment . Self-assessments and validation reviews placed one school at Performance 
Level 4, eight at Level 3, and one at Level 2. All ten schools provided reviewers with 
samples of site performance-based assessments and/or student portfolios. Since this is die 
first full year of implementation at most sites, schools indicated that complete results arc not 
yet available to evaluate student progress toward identified standards and expectations. 
Validation reviewers noted that teachers have played a central role in the development or 
selection of the assessments, and that they have been trained in the holistic scoring process at 
most schools. Student and parent involvement in the assessment process was cited by 
several sites and reviewers as an area for continued improvement 

Recognidon and Intervention . Validation reviewers rated three schools at Performance Level 
3 and seven at Level 2 in this area. All schools were able to identify student recognition and 
intervention activities which relate in some way to academic achievement, but their efforts for 
staff and parents are less well defined. At this time, rex;ognition and intervention practices for 
staff are largely informal, although some schools indicated they are exploring professional 
portfolios and alternative evaluation options. The primary issues placing schools more at 
Level 2 than Level 3 were the need to directly conelate recognition and inte?* Jntion practices 
to student achievement and school progress, and the neec* ic J-iyeHo^i and implement 
processes for each stakeholder group. 

Public Reporting . On this rubric, validation review results indicated that two schools were at 
Performance Level 4, four at Level 3, and four at Level 2. Schools rated highest were able to 
demonstrate they have multiple avenues to report out to the educational community, and to 
solicit input and involvement from teachers, staff, and parents. Those schools at Level 2 
indicated they have in place a variety of communication practices, but reviewers noted that 
these jctivities are not necessarily tied to the accountability system nor to student 
achievement. Several schools mentioned they plan to make changes in their school portfolio 
exhibition process and/or utilize the Annual Action Plan to help address the area of 
accountability reporting. 

CONCLUSIONS 

Based on the spring 1994 progress study and the fall 1994 self-assessment *uid validation 
review process, the ten LAD schools have continued to move forward in their accountability 
efforts. In the first project year, sites focused primarily on the identification of standards and 
related learner outcomes, and began investigating performance-based assessments to measure 
progress toward those outcomes. These efforts are reflected in the higher average ratings for 
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standards (3.1) and assessment (3.0). Duiing this second year, LAD schools have begun 
using their work in standards and assessment to identif / appropriate recognition and 
intervention strategies, and report on student achievement wid progress toward meeting 
school goals and expectations. Hie lower average ratings in recognition/intervention (2.3) 
and public reporting (2.8) parallel the schools' statements that, if these practices are to be 
based on student achievement and school progress, they must follow the development and 
implementation of site activities in standards and assessment 

LAD schools indicated that this self-assessment and validation review process was a valuable 
experience for the school and staff. Most r>ites noted that the accountability rubrics helped 
not only in assessing their current performance levels, but also in identifying the "next step" 
along each of the continuums to continue their efforts in implementing a stu^nt achievement 
accountability system. 

The five schools with new Comprehensive Site Plans reported that the Annual Action Plan 
section has provided a vehicle to help focus the school accountability efforts, including the 
delineation of leamer outcomes and related performance assessments, «»jid tiie provision of a 
process for specifying the roles and responsibilities of each stakeholder group. These 
schools also noted that the information gatliered through monitoring and evaluating the plan 
is expected to serve as one basis for recognition and intervention, and will become part of the 
accountability public reporting process. 

RECOMMENDATIONS 

1 . Promote the expansion of the accountability self-assessment process, including training 
for LAD Phase n and all other schools as they begin accountability implementation. 
Provide further training at the site and centra) office kvel to make self-assessment and 
validation review processes more meaningisal, comparable, and consistent. Include 
calibration in using the rabrics, direction for collecting and reviewing evidence, and 
strategies for involving all stakeholder groups and reaching consensus. LAD I school 
representatives might be trained as consultants and validation reviewers for other schools. 

2. Develop and disseminate suggested strategies to help sites make the *Viext step" in 
progressing along the accountability continuums, utilizing expertise from the LAD school 
sites that were rated at the higher performance levels on the accountabUity rubrics. This 
could be incorporated into the Comprehensive Site PUui training. 

3. Investigate ways to continue building assessment edacity at the site and district level in 
order to provide meaningful, consistent student achievement data which can be 
aggregated and disag^gated, and compared across schools in relation to the district 
standaj'ds and expectations. 

4. Provide support and training for schools in the identification and application of 
appropriate means of recognizing success, and the modification of programs and 
practices to intervene when progress is not made. 

5. Bring into alignment all accountability systems and requirements from national initiatives 
(Goals 2(XX), Title I, School-to-Work), state projects (Program (^u^ty Review, Western 
Association of Schools and Colleges (WASC) Accreditation, state assessments), and 
district activities (LAD, Comprehensive Site Plan, professional development, district 
assessment). Provide support and training at the site and central office levels to 
coordinate these programs and commit resources to the districtwide implementation of 
accountability. 
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ISSUE/CONCERN 

In June 1994, the board reviewed a Progress Report on the Leadership in Accountability 
Demonstration (LAD) Schools. Key findings from the study indicated that, at the time of 
data collection approximately six months into the accountability implementation process, the 
LAD schools were progressing in their work in the areas of standards and assessment but 
were only in the initial stages of addressing the elements of recognition/intervention and 
public reporting. After discussion of the progress study findings, the board requested 
additional study of the ten LAD schools as well as an interim report by January 1995 on the 
LAD school accountability efforts. This report presents findings from the first part of a year- 
long evaluation study of the original ten LAD schools. The interim study involved a school 
self-assessment and validation review process to examine the schools' status in each of the 
four elements of accountability as weU as the overall articulation and implementation of an 
accountability system. 

BACKfiROlJND 

In March 1993, the Report and Recommendations of the Student Achievement Accountability 
Committee (SAAC) was approved by the board. This report included a student acliievement 
accountability policy statement and identified four major constituent elements of an 
accountability system: standards, related assessments, recognition and intervention practices, 
and public reporting. The report also delineated a three-phase accountability implementation 
process (see Appendix A), beginning with the identification of ten demonstration schools that 
would provide leadership in developing an accountability system model for the district 
Training and initial planning occurred during the Developmen t and Assessment Phase in the 
spring of 1993 for the nine elementary schools and one middle school that applied to become 
Leadership in Accountability Demonstration (LAD) schools. The Resource/Assistance and 
Restructuring Phase was then designed to take place during the 1993-94 and 1994-95 school 
years. The SAAC report stated that progress would be assessed at the end of those two years 
using multiple indicators and review processes. The Renewal Phase would involve the 
implementation of recognition and intervention measures based on each school's progress in 
demonstrating accountability. 

The ten LAD schools received considerable training and support, beginning in the spring of 
1993 and continuing throughout the 1993-94 school year, to assess site needs and begin the 
articulation and implementation of an accountability system based on the four elements of 
accountability identified in the SAAC report. Each school was assigned a central office 
administrator as an advocate who would work with a site administrator to facilitate 
communication and collaboration between the school and central office. These site and 
central office advocates meet monthly as the Accountability Achievement Coordinating 
Committee (AACC) to support the LAD schools in the development and implementation of 
accountability systems. 
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In February and March of 1994, a progress study was conducted by the Planning, 
Assessment, and Accountability Division to examine what LAD schools had accomplished 
during the fust half of the 1993-94 school year to implement the four elements of 
accountability. Among the key findings from this study was the indication that LAD schools 
had dedicated considerable time to the identification of standards and learner outcomes, and 
to the selection and/or development of related assessments. However, the schools wei« still 
in the beginning stages of addressing the accountability elements of rccognition/mtervention 
and public reporting. This was to be expected, since recognizing, intervening, and reporting 
need to be based on student achievement information from site assessments which identify 
progress toward the standards and learner outcomes. 

Other district activities impacting accountability implementation also occurred during the past 
two years. Beginning with the 1993-94 school year, the district implemented a new 
comprehensive planning structure which was designed to incorporate processes to address 
the four accountability elements. Forty-five schools, including five of the LAD schools, 
wrote plans in this new format during 1993-94. The other five LAD schools were required 
to integrate the accountability elements into their existing single site plans until they enter the 
Comprehensive Site Plan cycle. In October 1994, thirteen additional schools were selected 
for LAD Phase H; these schools were among those who wrote Comprehensive Site Plans in 
1993-94 or who are writing plans in 1994-95. The original ten LAD schools have been 
partnered with the LAD II schools to help in the training and to provide support as the new 
schools begin developing accountability systems for their sites. 

One of the recommendations from the spring 1994 progress report called for a more formal 
study at the end of 1994-95 to assess accountability implementation at the LAD schools after 
two full years in the program. However, during board discussion of the progress report, it 
was requested that interim information be collected on the status of accountability 
implementation, especially regarding recognition and intervention. The board recommended 
that a work session be held in January 1995, focusing on accountability and the LAD school 
efforts. 

As a result of the board's request, the Planning and Accountability Team (PAT) developed an 
evaluation plan for the original LAD project, to be conducted in two phases during the 1994- 
95 school year (see Appendix B). The first part of the study occurred during the fall of 1994 
and involved site self-assessments of the LAD schools' current status in the implementation 
of the four elements of accountability. These self-assessment results were then validated by 
central office reviewers. The complete evaluation is proposed to mclude formal data 
collection at the end of the 1994-95 school year through interviews, surveys, observations, 
and document analysis, with the results presented to the board in September 1995. In 
addition, the District Achievement Accountability Committee (DAAC) will gather data during 
the 1994-95 school year to review each school's progress in demonstrating accountability, 
and will make recommendations to the board in the fall of 1995 regarding recognition and 
intervention measures. 

PURPOSE OF THE INTERIM STUDY 

The purpose of the first part of the evaluation study was to address the following questions: 

• To what extent have the LAD schools ailiculated and implemented an accountability 
system, and addressed each of the four elements of accountabiUty? 

• How do the schools rate their own achievement in each of these areas? 

• Are the school self-assessments validated by external reviewers? 
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The findings from this interim ««udy were also expected to identify exemplary practices 
which might assist other sch especially in the areas of recognition/intervention and 
public reporting. In addition, the study would serve to provide an assessment instrument for 
current and future use by the original ten LAD schools, the thirteen LAD Phase 11 schools, 
and all other district schools as they begin the implementation of accountability systems. 

METHODni/Y-.Y 

Development of the, F.val nation Tnstnitnp.nt 

In order to assess &e extent to which LAD schools have implemented the four elements of 
accoimtability, the Planning and Accountability Team proposed the development of a set of 
holistic evaluation rubrics. Site and central office advocates supported the idea at the 
September 1994 AACC meeting, and each LAD school administrator nominated one or two 
site staff members to serve on a rubric development committee. Chi SL mber 20, sixteen 
representatives from nine LAD schools (nine classroom teachers, three resource teachers, one 
instructional aide/govemance team member, one vice-princip^, and two principals) came 
together to draft rubrics for each of the four elements of accountability. Facilitated by PAT 
staff, the committee members examined in detail the original SAAC report, reviewed sample 
holistic rubrics and scoring processes, and shared site experiences. TSiey then divided into 
groups and drafted a four-point holistic evaluation rubric for each accountability element- 
standards, assessment, recognition/intervention, public reporting. After further discussion, 
the committee decided to add an additional rubric to address the overall articulation and 
implementation of a site-based accountability system. 

The draft rubrics were disseminated to the ten LAD schools and their central office advocates 
on September 30, with feedback requested by October 24. Revisions were made and the 
modified rubrics were sent out to the schools on November 1, 1994. Conceptually, the 
rubrics are actually continuums which range from descriptors of a school at the beginning 
stages of development (Level 1) to a school where there is fuU implementation and 
involvement of all stakeholders (Level 4). The complete rubrics are found in Appendix C, 
while Appendix D contains a compilation of all five rubrics into a summary matrix of the key 
statements for each performance level 

Data Collection 

Self-A.ssessment. The ten LAD schools conducted a self-assessment, using each of the five 
accountability rubrics, between November 1 and November 23, 1994. General directions for 
carrying out the self-assessment were presented at the October 21 meeting of the AACC, and 
were then disseminated by memo on November 1. Schools were asted to: 

• Review applicable site evidence, related to the implementation of accountability, with the 
staff, governance team, and other stakeholders; 

• Compare their evidence to the performance level descriptors for each rubric, and select the 
rating point which best reflects their assessment of the school's status at that point in time; 

• Complete a self-assessment rating sheet (see Appendix E), noting the performance levels, 
evidence/documentation reviewed, and any observations or comments. 

Validation Reviews. Staff meml)ers from the Pro-am Accountability and Consolidated 
Programs Services Units were assigned as validation reviewers; six resource teachers 
conducted one site visit each, while two specialists each reviewed two schools. Following a 
one-hour training session, the reviewers examined their assigned school's self-assessment 
and then arranged to visit the school between November 28 and December 7- Validation 
reviewers were directed to: 
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• Meet with site representatives to review the same evidence/documentation used by the 
school in determining its self-as,sessment ratings; 

• Validate the site rating, or recommend a different performance level, based on the evidence 
available and discussion with site representatives; 

• Complete a vahdation-review rating sheet, noting the performance levels, evidence 
reviewed, and any observations or comments; 

• Gather additional information regarding the involvement of the school's governance 
structure m the self-assessment, in the development of processes for recognition and 
mtervenuon, and in the LAD program overall 

At the November 8 meeting of the DAAC, the accountability rubrics and evaluation process 
were presented and members were invited to observe the validation reviews. Thirteen of the 
DAAC members were able to participate in a review visit 

Data Analysis 

Information on the self-assessment process was summarized from the self-assessment and 
vahdauon review rating sheets. Performance level ratings were tabulated and graphed for 
each of the five rubrics according to the number of schools placed at each rating point 
Average ratings were also determined for the self-assessment and validation review results 
for each rubric. The types of evidence reviewed, and the comments from site teams and 
validation reviewers, were analyzed holistically and summarized for each rubric. 

LMITATrnNS 

The short time available for planning and conducting this part of the study resulted in some 
mconsistencies in the self-assessment and validation review processes, and thus limitations m 
the comparability and generalizability of the findings. Due to the timeline, in part resulting 
from the year-round school fall vacation schedule, site teams did not receive specific training 
on conducting the self-assessment Therefore, schools took different approaches in 
gathering data and rating Uiemselves on the accountability rubrics, and involved staff and 
otfier stakeholders to varying degrees. Without previous experience in this process, and 
without more complete training on reviewing evidence and holisticallv assigning perfoimance 
leveLs, sites may have been inconsistent in their ratings. In addition! since the LAD schools 
were not told initially to collect and maintain specific documentation, sites have differing 
amounts and types of written evidence available to support their self-assessment ratings. 

Validation reviewers received only one hour of training before visiting sites for the validation 
reviews. Again, without experience and more complete training, reviewers may have been 
inconsistent in assigning performance ratings based on the available evidence and 
testimonials provided by the schools. Cases of disagreement between site and reviewer 
ratings were handled differently among reviewers. Some discussed differences with the site 
team and reached consensus on ratings, while other reviewers assigned different rating points 
witliout reaching consensus with the site. Each school was also given the choice of 
determining who from the site would be involved in the validation review visit, resulting in 
additional differences in the validation review process. 

KEYFTNDTMfiS 

Self-Assessment PrncRS S and Validation Rp.vip.w 

Sslf-AfisessmenlS. Five schools reported thai the self-assessment was conducted with input 
from or involvement of the teachers or the entire staff. Three of these schools had teachers or 
staff groups work on different rubrics and then share their recommended ra'dngs with the 



entire staff; the other two schools had teachers individually recommend ratings which were 
then compiled by the administrator or LAD team. At four schools the LAD site team met 
together to determine the self-assessment ratings, while one school had a different site 
committee assess each of the rubrics. In all cases, the sites indicated that final ratings were 
shared with the staff and the appropriate governance groups. During debriefing at an AACC 
meeting, representatives from most of the schools reported that they found the rubrics to be 
very useful tools, and the process a valuable one for the school and staff. 

Schools reported using a variety of evidence and documentation to determine their self- 
assessment ratings. Types of evidence included student portfolios and portfolio menus or 
checklists; site-developed learner outcomes and observable behaviors; Annual Action Plans; 
performance assessments and rubrics; minutes and agendas from committee meetings, 
governance groups, staff development sessions, parent activities; newsletters, brochures, 
school handbooks, School Accoimtability Report Cards; school portfolios and documentation 
from exhibitions. 

Validation Reviews . At two schools, the reviewer met with a site administrator (principal or 
vice principal) to conduct the validation review. The review visit at three other schools 
involved the principal, along with the vice principal and/or site resource teacher. At the 
remaining five schools, one or more classroom teachers joined the adniinistrator(s) for the 
validation review visit. Nine of the ten validation review visits were obser/ed by one or two 
DAAC members. Validation reviewers reported that the site visits ranged in Jength from one 
to two hours. 

Validation reviewers at three schools indicated that much of the site evidence was provided 
orally, through testimonials from the participating staff. In contrast, reviewers for four other 
schools noted that extensive documentation was made available on-site. During a debriefing 
of the process, three validation reviewers reported that the site representatives appeared 
uncomfortable and nervous about the validation review process, and seemed challenged to 
prove themselves in trying to justify the self-assessment ratings. In two of these cases, the 
reviewer expressed disagreement with the site rating on one or more of the rubrics; after 
discussion, one school d^d change the self-assessment ratings to agree with the validation 
reviewer. 

Accountability Performance Leveis 

Table 1 summarizes the self-assessment and validation review ratings for each of the five 
accounlability rubrics. Overall, there was agreement about 86 percent of the time between the 
validation review and the self-assessment rating; in 6 percent of the cases the validation 
reviewer recommended a higher rating, while in 8 percent the reviewer's rating was lower 
than tlie self-assessment level, gtandards had the highest average ratings with most schools 
placed at Performance Levels 3 and 4. The next highest ratings were for Assessment and 
then Articulation and Implementation of an Accnuntahilitv Svstem. Slightly lower average 
ratings were found for puhljc Reporting and then Recognition/Intervention : in these areas, 
most schools were at Performance Levels 2 and 3. No school was rated at Level 1 in any 
area, while validation reviewers assigned four ratings at Level 4 (one in standards, one in 
assessment, and two in public reporting). 
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Number of Schools at Each Performance Level and Average Performance Level Ratings 
Self-Assessment and Validation Review Results 



Perf. 
Level 


Articulation & 
Implementation 


Standards j Assessment 


Recognition & 
Intervention 


PubUc 
Reoortin g 




Self 


i Valid. 


Self 1 Valid 


Self 


Valid, 


Self 


j Valid. 


Self 


Valid 


4 


1 


0 


2 I ; 


1 


J 


0 


0 


1 


V 

2 


3 


7 


9 


f 

7 1 9 


8 


8 


4 


3 


6 


4 


2 


2 


1 


1 1 0 


1 


1 


6 


7 


3 


4 


1 


0 


0 


t 

0 1 0 


0 


0 


0 


0 


0 1 


0 


Average 


2.9 


2.9 


3.1 1 3.1 


3.0 


3.0 


2.4 


2.3 


: 

2.8 


2.8 



ArQcul a aon and Implementation of an Accountahilitv Sy .^tpm Hie Accountability System 
Guiding Principles from the SAAC report were used to help develop the overall articulation 
and implementation rubric. A Level 4 school in this area has fuUy implemented a site 
accountability system, including integrating accountability into school planning processes 
witii Ihe involvement of all stakeholder groups. The accountability system is responsive to 
the identified needs of students, and includes all the constituent elements. Expectations for 
student achievement have been articulated to all stakeholders and provide tiie focus for 
teaching and learning; stakeholders know their roles and demonstrate their responsibilities for 
student achievement An ongoing self-review process is in place through which tl^ 
govemance team coordmates tiie evaluation and communication of student achievement and 
school progress toward goals and expectations. 

Figure 1 summarizes tiie results of the self-assessment and validation review ratings for this 
overall articulation and implementation rubric. According to tiie self-assessments, one school 
placed itself at Performance Level 4, seven schools were identified at Level 3, and two 
schools rated tiiemselves at Level 2. Eight of these ten ratings were validated by tiie 
reviewer; in tiie two cases of disagreement, one reviewer recommended a higher rating (Level 
3 rather than tiie site-assigned Level 2), while in tiie otiier case tiie reviewer gave a lower 
rating (Level 3 ratiier than the site's Level 4). 
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Figure 1. Validation Review and Self- Assessment Ratings for 
Articulation and Implementation of an Accounta bility System 



To what extent have the LiD schools articulated and implemented an accountability system? 

The schools and validation reviewers referenced a wide variety of evidence to support their 
ratings in this area. Comments on Level 3 schools included a note from one reviewer that the 
site seems "seriously committed to the accountability process," while another reported that 
the school is "in the process of moving from informed to involved on an ongoing basis." 
Still another reviewer wrote that the school "seems to have a handle on where they are and 
where they need to go next" At five schools, the reviewers indicated the site is developing 
or refining a self-review process to monitor accountability implementation, evaluate school 
progress and student achievement, and communicate results to the educational community. 
Five of the LAD schools have new Comprehensive Site Plans, and all of them particularly 
referenced the Aiuiual Action Plan section as assisting their processes for the articulation and 
implementation of accountability. 

Weaknesses were primarily identified in the area of stakeholder involvement. Both the 
schools themselves and the validation reviewers reported that, while most teachers seem to be 
aware of their roles and responsibilities relative to accountability for student achievement, 
there has been less articulation to and involvement of other staff, parents, and students. 

Standards . The Report and Recommendations of the Student Achievement Accountability 
Committee indicated that "the accountability process at each school will be correlated with a 
district system of high standards." Schools at Level 4 in this area have identified standards 
and prioritized them based on site-identified needs. Learner outcomes and observable 
behaviors have been collaboratively developed based on the standards, and have been cleariy 
articulated to the entire educational community. Professional development focuses on 
improving teaching practices and adjusting instruction to meet the diverse needs of the 
student population. 

As can be seen in Figure 2, the ratings in the area of standards differed slightly between die 
self-assessments and validation reviews. After conducting the self-assessment, two schools 
rated themselves at Performance Level 4, seven were identified at Level 3, and one gave itself 
a rating at Level 2. Eighty percent of these ratings were validated by the reviewers; in one 



case, the validation reviewer assigned a higher rating than the self-assessment (Level 3 rather 
than Level 2), while in another case the reviewer felt Level 3 was more appropriate than the 
site-designated Level 4. 
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Figure 2. Validation Review and Self-Assessment Ratings for Standards 



Have the LAD schools identified high standards for student achievement? Have learner 
outcomes and accompanying performance expectations been developed? 

All ten schools have identified standards and developed related learner outcomes or grade 
level expectancies. Evidence included site Annual Action Plans with selected standards, 
learner outcomes, and key observable behaviors; grade level expectancies brochures; 
developmental continuums; portfolios and portfolio checklists; and agendas from staff 
development activities. Schools reported using the state curriculum frameworks, draft 
district standards, district goals and expectations, New Standards Project materials, and 
documents and information from other states and districts, in identiiying standards and 
developing their learner outcomes. 

At all schools, the focus so far has been in language arts, as per the district emphasis. Some 
schools have begun addressmg mathematics, while one school has extended the process to 
other content areas as well. Several sites reported that they began work in this area before 
becoming LAD schools, and in some cases the school terminology is not yet fully aligned 
with the new district, state, and national language. Most schools reported that the learner 
outcomes have been developed collaboratively by teachers with at least some involvement of 
other stakeholder groups. 

Self-assessment and validation review conmients indicated fliat efforts arc continuing at all 
schools to articulate the outcomes and expectations to the entire educational community, and 
to further involve all staff, parents, and students. Reviewers at two schools noted that staJFf 
turnover and a highly mobil'5 community have required a ccHnmitroent to on-going 
development and articulation in this area. 
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Assessment. The SAAC report recommended that a variety of assessment instruments and 
data sources be employed to evaluate student and school performance in light of adopted 
standards, and that on-going self-evaluation should be a key component of any assessment 
system. A school at I^evel 4 would have a site assessment system in place in at least one area 
of focus, and be in the process of development and implementation for other academic areas. 
The Level 4 school is utilizing multiple forms of assessment to identify student progress 
toward the learner outcomes and standards, and students are provided with a variety of 
opportunities to demonstrate what they know and are able to do. Teachers meet regularly to 
examine student work, and they utilize assessment results to modify the teaching and learning 
process. Students are also actively involved in examining and assessing their own work. 

Figure 3 summarizes the ratings on the assessment rubric. In this area there was iOO percent 
agreement between the self-assessment and validation review ratings. One school was rated 
at Performance Level 4, eight schools were placed at Level 3, and one school was assigned 
to Level 2. 
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Figure 3. Validation Review and Self -Assessment Ratings for Assessment 



What performance-based assessments are being utilized at Jffte LAD schools? Do ike 
assessments measure student progress toward the site-identified standards and learner 
outcomes? When will the assessments be administered and how will they be scored? 

All ten schools provided the validation reviewers with samples of site performance-based 
assessments and/or sample student portfolios. The majority of the evidence reflcjcted 
assessments in the area of Irnguage arts, again in accordance with the district emphasis. 
Several schools noted that this is the first ^11 year of implementation of new assessment 
systems, so results may not be available yet to evaluate student achievement and school 
progress toward identified standards and expectations. However, most schools indicated 
they plan to administer assessments and/or evaluate portfolios on a schoolwide basis between 
two and four times during the year. 

Eight schools reported they are using holistic scoring rubrics to evaluate student work, and 
that teachers have been trained in the holistic scoring process. At six sites, the self- 
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assessment and/or validation review infonmation indicated that teachers meet collaboratively 
to examme student work or review assessment results. Seven of the validation reviews noted 
that teachers have played a central role in the development and/or selection of the 
assessments, and most also noted that schools have made a variety of efforts to share 
mformation about the assessments and the results with parents. Six schools reported that 
students have at least some involvement in the assessment process; this participation ranged 
from stadent selection of portfolio items, to students scoring their own work and that of 
peers. Student and parent involvement in tiie assessment process were cited by several sites 
and reviewers as an area for continued improvement 

RgCOgnition and IntPrvention. The SAAC report indicated that "recognition and intervention 
practices will be used to acknowledge student achievement and to assure ^propriate 
consequences if a school does not achieve its goals and objectives." A school at Level 4 is 
miplementing a recognition and intervention system for all stakeholders (students, staff, 
parents, community) that is directiy related to achievement of the school's identified 
standards and learner outcomes, and the district goals and expectations. There is a clearly 
defined, on-going self-review process in place, and stakeholders can articulate their roles and 
responsibilities for student achievement. The system identifies and provides appropriate 
recognition to stakeholders who contribute to improved student achievement and also 
identifies and provides appropriate support for stakeholders requiring assistance. 

In addressing tiiis accountability element, die rubric development committee felt it necessary 
to further delineate between recognition and intervention, noting that schools may be farther 
along in one area tiian the otiier. Therefore, schools and validation reviewers were asked to 
identify performance levels for recognition and intervention separately, as well as assign an 
overall rating. Figure 4 represents tiie overall ratings for recognition and intervention. Based 
on tiie self-assessment results, four schools assigned tiiemselves to Performance Level 3 and 
six schools placed tiiemselves at Level 2. Validation reviewers agreed with tiiese ratings at 
nine of tiie ten schools; tiie otiier school was assigned a rating at Level 2 rather tiian tfie site- 
selected Level 3. 
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Figure 4. Validation Review and Self- Assessment Ratings for Recognition and Tntp.rventinn 
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Eight of the ten schools rated themselves at the same level in both recognition and 
intervention (f ur at Level 3 and four at Level 2). Seven of these eight ratings were validated 
by the reviewer; in one case, the reviewer raised the school from Level 2 to Level 3 in the 
area of intervention. At the two schools that assigned different performance levels in 
recognition and intervention, both considered themselves to be performing at Level 3 in 
recognition, but at Level 2 in intervention and more at a Level 2 overall. 

How will a recognition and intervention system be implemented for stakeholders? What will 
happen when assessment information shows progress is or is not being made toward the 
standards and learner outcomes? 

Schools provided primarily verbal reports and testimonials on student, staff, and parent 
recognition and intervention practices in place at their sites. One school has prepared 
checklists which identify recognition and intervention activities for each stakeholder group; 
the site reported that these actions are correlated to student achievement in language arts. 
Another site has developed extensive lists of student intervention and recognition strategies, 
as well as brief lists for staff and parents. Four schools mentioned that they have just titis 
year completed the identification of specific learner outcomes (either through tiie Annual 
Action Plan or other site activities) and are now implementing schoolwide assessments 
correlated to those outcomes. These schools reported tiiat they are currently in the process of 
specifying recognition and intervention measures to apply when the data from these 
assessments are available at the end of this school year. 

All schools were able to cite student recognition and intervention activities which relate in 
some way to academic achievement. Most schools said that students are recognized in 
classroom and schoolwide activities for academic accomplishments; all schools could also 
reference student intervention processes, many of which are based in the site consultation 
team concept. Four schools noted that student/teacher/parent contracts are being used as part 
of tiie recognition and intervention system. However, validation reviewers indicated that 
additional development is needed to more directiy relate these contracts to student 
achievement and school goals, and to identify processes for follow-up when one or more 
parties do not carry-out their part of the contract 

Implementing meaningful, appropriate recognition and intervention measures for staff 
continues to be an area of concern for schools. Two sites reported on tiie use of professional 
portfolios in relation to teacher recognition and intervention. Four schools also mentioned 
the alternative evaluation options for certificated staff Otherwise, staff lecognition and 
intervention procedures were reported as largely inforraal. Recognition activities listed site 
bulletins, administrative and peer feedback, and invitations to malre conference or workshop 
presentations. Intervention strategies included administrative feedback and recommendations 
for professional development One reviewer noted that tiie site has "few formal options for 
providing recognition and intervention for teachers and other stakeholders," a perspective 
reported by several other validation reviewers as well. 

The primary issues placing schools more at Level 2 than Level 3 were the need to directiy 
correlate recognition and intervention practices to student achievement, and the need to 
develop and implement processes tiiat apply to all stakeholders: stiidents, teachers, otiier 
staff, parents, and the community. One reviewer noted that tiie site indicated "!iiey need to 
develop a more consistent metiiod for recognition and intervention that correlates to tiie 
accountability plan." Aiiotiier school wrote in its self-assessment tiiat they focus on 
"recognizing students and staff members for participation, honors and achievements, and 
working collaboratively" but are just "beginning to ^gn our recognition and intervention 
measures witii student achievement." 
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Public Rsportlng. One of the guiding principles in the Report and Recommendations of the 
Student Achievement Accountability Committee stated that "schools will fully and accurately 
report student achievement results in a public process that emphasizes progress achieved." 
The Level 4 rubric descriptors call for student achievement to be the primary focus of the 
accountability reporting process, and for accountability reporting to an integral part of 
school planning. At this level, there is a comprehensive understanding among all 
stakeholders of goals, expectations, and responsibilities, and there is a well-defined two-way 
system of communication among the stakeholders about student achievement and progress 
toward those goals and expectations. The school also has in place an on-going self -review 
process to evaluate and communicate school performance in accountabiUty for student 
achievement 

Figure 5 provides a graphic display of the results in public reporting. In the self-assessment 
results, one school placed itselSf at Performance Level 4, six schools were at Level 3, and 
three schools assigned themselves to Level 2. Eight of these ratings were validated by the 
reviewers, while two reviewers recommended different ratings-one higher (Level 4 rather 
than the site-assigned Level 3), and one lower (Level 2 over the site-selected Level 3). 
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Figure 5. Validation Review and Self- Assessment Ratings for Public Reporting 



How will schools report school progress and student achievement to the educational 
community? Will die school portfolio/exhibition show evidence of progress toward the 
standards and learner outcomes ? 

Schools referenced a wide variety of evidence when reporting on their performance in this 
area, but validation reviewers indicated that most of the information was shared orally. Some 
schools did show materials such as sample parent bulletins, newsletters, and brochures; 
minutes and agendas from meetings and inservice activities; student handbooks; charts and 
lists of committees and governance groups. The two schools rated at Level 4 were able to 
demonstrate that they have multiple avenues through which they report out to the educational 
community; they also regularly solicit input and involvement from teachers, staff and parents. 
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Schools rated at Level 2 indicated they have a variety of reporting activities, but these are not 
necessarily tied to the accountability elements nor to student achievement. Communication at 
these sites was also primarily one-way, from school to home, through parent conferences 
and traditional activides such as Open Houses. Sites rated at Levels 2 and 3 identified several 
ideas to help improve in this area, including using student portfolios and exhibitions in public 
reporting activities; connecting repnrf lng to the Annual Action Plan learner outcomes ai i' 
observable behaviors; modifying the School Accountability Report Card to reflect the 
accountability elements; and having the LAD team and/or governance team take leadersliip in 
commimicating school peiformance. 

Accnuntahilitv and School nnvemance Structures 

Validation reviewers were asked to gather additional information from the schools regardtr.g 
the school governance structure in relation to the accountability self-assessment, development 
and implementation of a recognition and intervention system, and the overall LAD project 
All schools indicated that their accountability efforts are primarily coordinated by a site LAD 
team, or an Assessment and Accountability Team if school committees arc organized aroimd 
the five Design Tasks. The core of this LAD team or accountability committee is usually the 
site leadership team that participated in the original training sessions. According to the 
schools, this accountability committee informs and consults with the site governance team on 
a regular basis; in most cases one or more members of the accoimtability committee arc also 
on the governance team. One school reported that, beginning with the 94-95 school year, the 
school governance team has become the LAD team; the committee is looking into developing 
a rubric to monitor school programs and assess the school's accountability efforts. At 
another school, the govemance team by-laws were modified to writs in a responsibility for 
overall accountability as related to the district expectations. 

At all schools, the site govemance team was informed about die self-assessment process and 
advised of the results. One site had the govemance team rate the school on the overall rubri 
for Articulation and Implementation of an Accountability System. Reviewers at two other 
schools reported that the self-assessment was conducted by the staff or accountability team, 
with Ae results presented to the govemance team for approval by consensus. At four 
schools, staff members who were also govemance team members participated in completing 
the self-assessment. For the remaining three schools, the results were reported to the 
govemance team after the self-assessment was completed by the accountability team or staff. 

At most LAD schools, the staff and/or accountability committee have primary responsibility 
for development and implementation of the recognition and intervention system. Governance 
teams have been kept uiformed but most have not been directly involved. One reviewer made 
a comment that seemed to be representative, indicating that the "school is still in the process 
of developing and articulating the govemance team's role in this area." 

rONCLUSIONS 

The ten Leadership in Accountability Demonstration schools have continued to move forward 
in their accountability efforts. To assess the extent to which these schools have implemented 
an accountability system, self-assessments and validation review visits were conducted 
during November and December 1994, using four-point holistic evaluation rubrics. The 
highest performance ratings were in standards and assessment, with most schools placed at 
Performance Level 3. Schools were mostly at Levels 3 and 2 in public reporting, and more 
at Level 2 in recognition and intervention. Validation reviewers assigned four Level 4 
performance ratings-one in standards, one in assessment, and two in public reporting. 
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Implementation nf Arrnnnti^l^jjjty 



ArQgul a t j On and Imnlgmentation of an Accountflhility Systp.m The results of the spring 1994 
progress study and the fall 1994 interim study indicate that LAD schools focused in the first 
year on the identification of standards and related learrer outcomes. The schools then began 
investigating and implementing perfomiance-based assessments to measure progress toward 
attainment of these site-idendfied outcomes. During this second year, schools have begun 
usmg their work in standards and assessment to identify appropriate recognition and 
mtervention strategies, and report on student achievement and progress toward meeting 
school goals and expectations. 

Standards and Assessmgnt- Schools have continued to expand and refine their efforts in the 
areas of standards and assessment. In the spring 1994 progress study, aU LAD schools at 
least mentioi.ed standards, and seven of the ten discussed the development of learner 
outcomes and benchmark indicators. At that time, the schools were also in the process of 
unplementing portfolio systems and/or performance-based assessments. By the fall of 1994, 
the LAD schools all had learner outcomes or grade-level expectations in place, at least in 
language arts, which they indicate have been developed from distiict standards and 
expectations as well as the state frameworks. The schools arc also weU on the way to 
implementing language arts performance-based assessments and portfolios on a schoolwide 
basis. Teachers are starting to examine student work collaboratively, and involve students in 
the assessment process. 

RgCPgnitiPn and IntervgntiPJl- The spring 1994 progress report noted that those school 
representatives who even addressed the issue of recognition and intervention related it 
primarily to students, but not the other stakeholders identified in the SAAC report Fall 1994 
data indicate that schools have begun focusing on the areas of recognition and intervention, 
although they remam at different levels in this process. At tiiis time, it appears that schools 
have a better understanding of the concept of recognition and intervention. The primary issue 
has become the investigation of ways to meaningfully connect recognition and intervention 
processes to the role of each stakeholder group in student achievement and the 
accomplishment of the school's goals and expectations. Several schools stiessed that 
recognition and intervention practices should follow tiie development and implementation of 
site activities in standards and assessment. 

Publig Rgp^rting- Based on the spring 1994 progress study, most of the reporting activities 
identified by the LAD schools seemed to be primarily one-way, from school to parents, and 
may or may not have been related to progress toward meeting learner outcomes and school 
goals. Only two of the schools had mentioned reporting to stakeholders on the whole school 
program. By the time of tiiis fall 1994 self-assessment process, nearly all schools indicated 
that they have schoolwide exhibitions as well as other staff, parent and community activities 
for disseminating information and promoting participation. While some schools were able to 
articulate that thoir public reporting activities are aligned with the accountability system and 
student achievement, otiier schools have not yet flilly made that correlation. At all schools, 
however, efforts continue to keep the educational community informed about school activities 
and involved in school events. 



Integration of Acconntahilitv into School Planning Prnrp-wps 

At the five LAD schools with new Comprehensive Site Plans, these plans were refwenced 
throughout the self-assessment and validation review listings of evidence, as well as in the 
comments and observations. One school self-assessment and validation review identified the 
plan as evidence for the rating in each of the five accountability rubrics. These five schools 
indicated that the Annual Action Plan has provided a vehicle to help focus the school 



accountability efforts, including the delineation of learner outcomes and related performance 
assessments, and the provision of a process for specifying the roles and responsibilities of 
each st^eholder group. These schools also noted that the information gathered through 
monitoring and evaluating the plan is expected to serve as one basis for recognition and 
intervention, and will also become part of the accountability public reporting process. 

Accountability Rubrics an d School Self-Assessment 

Most of the LAD schools indicated tliat this self-assessment and validation review process 
was a valuable experience for the site. Schools reported that the rubrics helped not only in 
assessing their current perfonnance levels, but also in identifying the "next step" along each 
of the accountability continuums. One site requested that a list of strategies be developed to 
identify practices and processes which coujd help a school move along the continuum to the 
higher performance levels. The LAD Phase n schools also used the rubrics at their initial 
training to help in conducting the school needs assessment. 

RECOMMKNDATIONS 

1. Promote the expansion of the accountability self-assessment process, including training 
for LAD Phase n and all other schools as they begin accountability implementation. 
Provide further training at the site and central office level to make self-assessment and 
validation review processes more meaningful, comparable, and consistent. Include 
calibration in using the rubrics, direction for collecting and reviewing evidence, and 
strategies for involving all stakeholder groups and reaching consensus. LAD I school 
representatives might be trained as consultants and validation reviewers for ottier schools. 

2. Develop and disseminate suggested strategies to help sites make the "next step" in 
progressing along the accountability continuums, utilizing expertise from the LAD school 
sites that were rated at the higher performance levels on the accountability rubrics. This 
could be incoiporated into the Comprehensive Site Plan training. 

3. Investigate ways to continue building assessment capacity at the site and district level in 
order to provide meaningful, consistent student achievement data which can be 
aggregated and disaggregated, and compared across schools in relation to the district 
standards and expectations. 

4. Provide support and training for schools in the identification and application of 
appropriate means of recognizing success, and the modification of programs and 
practices to intervene when progress is not made. 

5. Bring into alignment all accountability systems and requirements from jiational initiatives 
(Goals 2000, Title I, School-to-Work), state projects program Quality Review, WASC 
Accreditation, state assessments), and district activities OLAD, Comprehensive Site Plan, 
professional development, district assessment). Provide support and training at the site 
and central office levels to coordinate these programs and conunit resources to die 
districtwide implementation of accountability. 
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APPENDIX A 
ACCOUNTABILrrY PROCESS OVERVIEW 



A-1 



2G 



Accountability Process Overview 
Phase 1: Development and Assessment Phase (February-June 1993) 

1. Applications submitted by school sites interested in becoming Leadership in Accountability 
Demonstration (LAD) schools. 

2. SAAC and area assistant superintendents select 10 demonstration schools from among applicants. 

3. Demonstration schools conduct assessment of site needs. District Assistance Teams formed on tiie 
basis of site needs. 

4. With assistance from the SAAC, demonstration schools, subject to the consensus agreement of their 
site governance teams, and District Assistance Teams will develop an accountabf lity process. The 
process will include recognition and intervention plans and may be used as a model for a dis- 
trictwide accountability system. The accountability process will be articulated to all stakeholders and 
integrated into existing planning processes at the LAD school sites. Waivers will be requested as 
needed. 

5. The District Standards Board will develop district standards in language arts and mathematics. 
Those standards vdll be articulated to the demonstration schools and to all stakeholders. 

6. Demonstration schools will redirect resoiu'ces and staff to meet flie sites' accountability objectives. 

7. Training v\nd staff development will be provided at demonstration schools in developing standards, 
learner outcomes, alternative assessment and performance content. Training of stakdiolders will be 
directly relattrl to their responsibility in improving student achievement. 

Phase 2: Resource/Assistance and Restructuring Phase (September 1993-June 1995; 
timeline adjustable) 

1. With the help of District Assistance Teams and other district resource personnel, demonstration 
schools implement accountability within their existing site plans. 

2. Training and staff development at demonstration schools and training of stakeholders continues. 

3. Development of standards and leamer outcomes continues. 

4. Progress or lack of progress is assessed by review of multiple indicators and of findings of existing 
review processes. Site governance teams, with support from District Assistance Teams, evaluate 
results. 

5. Site govemarKe teams and the District Assistance Teams refer findings of the review process to the 
District Accoimtability G)mmittee. 

Phase 3: Renewal and Development/Assessment Phase (1995- ) 

1. Recognition for progress or intervention for lack of progress implemented; implementation shall not 
conflict with state law, school district policy or employee contracts. 

2. In the 1994-1995 school year, all district schools will begin the Development/ Assessment Phase and 
subsequently will proceed throu^-i the second and third phases. 
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APPENDIX B 
1994-95 LAD EVALUATION PLAN 
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Evaluation of the 
Leadership in Accountability Demonstration (LAD) Schools 
Two-Part Study: 1994-95 



Purpose of Study 


Proposed Process 


Timeline 


Interim report on the implementation of 
accountability at the ten original LAD 
schools, to provide: 

• performance level ratings for each 
accountability element, with comments 
and observations about site activity 

• information/"exemplars" for LAD I 
and n schools 


• Development of holistic scoring rubric for 
each of the accountability elements 
(committee of LAD site representatives) 

• School self -assessment using holistic 
rubrics 

• Validation review by evaluators (PAA 
staff) with DAAC observers 


September 1994 

November 1994 

December 1994 

Board Report: 
January 1995 








Evaluation of the first two years of the 
LAD I project, to identify: 

• implementation process, including 
integration of accountability into 
school planning 

• involvement of all ctakeholder groups: 
teachers, other staff. Gov. Team/SSC, 
parents, students, community 

• initial effects of accountability 
implementation on: student 
achievement, instructional practices, 
staff morale, parent/commimity 
involvement, school climate, etc. 

• factors that contributed to or impeded 
implementation of accoimtability 

• resources, support used/needed 

• LAD schools assistance to other 
schools (LAD n and beyond) 


• Group interview of site LAD team 

• Survey of: teachers, other staff, parents 

• Review of site documentation: site plans, 
assessments, staff development agendas, 
governance team minutes/agendas, etc. 

• Group interviews: teachers, other site 
staff, parents, students, governance team; 
advocates 

• Observations: classroom/schoolwide 
activities identified by sites 

• Review of student achievement data: 
—site based assessments 

-test results (as available/applicable) 

• Review of 16 Expectations' performance 
indicators 

Evaluators: 

• PA/PAA staff for tea;her/staff interviews 
and site observations 

• Consultant for parent contacts 


April 1995 
April-May 1995 
April-May 1995 

May 1995 

May 1995 
June 1995 

June 1995 

Board Report 
September 1995 
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APPENDIX C 
ACCOUNTABILrrY RUBRICS 




AETH^CIOLAM^N AN© HMPLISMISOTAirnON m AN 
ACCOIONITAIBIILnirY SYSTISM 

Level ^ 

The school has fully implemented their site-developed accountability system. Accoxintability has 
^n completely integrated into school planning processes with the involvement of all stakeholder 
groups. The accountability system is responsive to the identified needs of students, and includes the 
following elements: student achievement goals and outcomes based on state/district/site standards, 
appropriate means of assessing attainment of the outcomes, a set of appropriate recognition and 
intervention measures, and public reporting practices. Expectations for student achievement have 
been articulated to all stakeholders and provide the central focus for teaching and learning. 
Stakeholders know their roles and demonstrate their responsibilities for student achievement. An 
ongomg self-review process is in place through which the governance team coordinates the 
evaluation and communication of student achievement and the school's progress toward meeting the 
identified goals and expectations. 

Level 3 

The school has used key documents* to develop a site accountability system. Accountability has 
been integrated into school planning processes with the involvement of most stakeholders. The 
accountability system includes student achievement goals and outcomes based on state/district/site 
standards, appropriate means of assessment, a set cf recognition and intervention measures, and 
public reporting practices. Expectations for student achievement have been clearly articulated and 
most stakeholders consistently use them to focus tei. ,hing and learning. Stakeholders know their 
roles and generally demonstrate their responsibilities for student achievement The governance team 
has implemented a self-review process to coordinate tlie evaluation and communication of student 
achievement and the school's progress toward meeting the identified goals and expectations. 

Level 2 

The school is in tlie process of developing an accountability system and integrating it into school 
planning processes. Stakeholders are examining key documents* and beginning to: select standards 
and develop learner outcomes, identify and/or develop related assessments, develop a system of 
recognition and intervention, and make plans for conmiunication and public reporting. Some 
expectations for student achievement have been articukited, but may not be consistency used to 
focus teaching and learning. Some stakeholders kno their roles and responsibilities for improving 
student achievement The governance team is beginning to develop a self-review process to evaluate 
and communicate school performance in accountability for student achievement 

Level 1 

The school is becoming aware of the elements involved in an accountability system, and the need to 
integrate accountability into school planning processes. Stakeholders are being identified, and the 
school is in the process of conducting an accountability needs assessment Roles and 
responsibilities, and expectations for student achievement, are known to some stakeholders. The 
governance team is discussing accountability, and beginning to determine its role in tlie 
implementation of the accountability system. 



*Key documents may include: Task Force Reports-Zri Elementary, Caught in the Middle, Second 
to None; California State Curriculum Frameworks; district goals and expectations; national, state, 
and district standards. 
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SITANIDAIEIDS 
(Content, Performance, Opportunity to Learn) 

Level 4 

The school has identified standards, and prioritized the standards based on site-identified needs. 
Learner outcomes and observable behaviors have been collaboratively developed based on the 
standards, district goals and expectations, and the school's selected aiea(s) of focus. The 
standards, learner outcomes, and observable behaviors have been clearly articulated to the entire 
educational community. Staff members are fully involved in professional development that 
focuses on improving teaching practices as they relate to standards and learner outcomes. On an 
ongoing basis, staff members review the learner outcomes and observable behaviors and adjust 
instructional practices to meet the diverse needs of their student population. Standards, learner 
outcomes, and the related assessments serve as the basis for recognition and intervention to ensure 
progress in student achievement 



Level 3 

The school has identified standards based on site needs, and collaboratively developed learner 
outcomes and observable behaviors based on the standards, district goals and expectations. The 
standards, learner outcomes, and observable behaviors have been articulated to most of the 
educational conmiunity. Professional development has been designed to focus on improving 
teaching practices as related to the standards and learner outcomes. Most staff members regularly 
review the learner outcomes and observable behaviors and adjust instructional practices to meet the 
needs of the student population. Standards, learner outcomes, and the related assessments 
generally serve as the basis for recognition and intervention. 



LmL2 

The school is in the process of examining site needs, identifying standards based on those needs, 
and developing learner outcomes and observable behaviors based on the standards, district goals 
and expectations. Stakeholders are participating in professional development that focuses on the 
process of identifying standards, developing learner outcomes, and improving instructional 
practices. 



Level 1 

The school is examining key documents relating to standards (e.g., Task Force Reports-/f',$ 
Elementary, Caught in the Middle, Second to None, California State Frameworics and Model 
Curriculum Guides, national and district standards). Stakeholders are learning the terminology 
(e.g. standards, learner outcomes, observable behaviors), and the staff is engaging in dialogue 
related to standards. 
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Level 4 

(The site assessment process is in place at least in the school's area of focus, and is in the process 
of development and implementation for other academic areas.) 

Multiple forms of assessment are utilized in identifying student progress toward the learner 
outcomes and standards. Students have a variety of opportunities and contexts in which to 
demonstrate what they know and can do in relation to the learner outcomes. Teachers play a 
central role in the development and/or selection of the assessments. Teachers meet regularly to 
collaboratively examine student work and improve the teaching and learning process. Assessments 
arc reviewed and adjusted to meet the needs of the diverse student population. Materials and 
resources arc available for effective instruction and assessment Students are actively involved in 
examining and assessing their own work. Stakeholders use information from the assessments, as 
well as a variety of other indicators, to evaluate the educational program and inform the recognition 
and intervention process. 



Level 3 

(The site assessment process is in place at least in the school's area of focus, and is in the process 
of development and implementation for a second focus area,) 

Several forms of assessment are utilized in identifying student progress toward the learner 
outcomes and standards, and students have various opportunities to demonstrate what they know 
and can do. Teachers are involved in the development and/or selection of the assessments, and the 
school is piloting and revising assessment measures. Teachers meet to colkboratively examine 
student work, and modify the teaching and ieaming process. Assessments are adjusted to meet the 
needs of the student population. Students eie involved in examining and assessing their own 
work. Assessment information is generally used to evaluate the school program and inform Jhe 
recognition and intervention process. 



LmL2 

(The site assessment process is in development in the school's area of focus.) 

The school site is researching performance assessments through Ic king at the Task Force Reports 
(It's Elementary, Caught in the Middle, Second to None), the State Curricular Frameworks, 
national and district standards, other schools' assessment measures, and on-site assessments 
already in place. Grade level/department/committee meetings are being held to identify school 
population needs and select or develop appropriate performance-based assessments correlated to 
the site-identified learner outcomes. The school is beginning to establish a collaborative 
atmosphere for ieschers to examine student work, and students are beginning to be involved in the 
examination and assessment of their own work. 



Level 1 

The school site has selected an area of focus, and is beginning to investigate various assessments 
and how they relate to standards and learner outcomes. The site is beginning to identify needed 
inservice baining in the area of assessment, and is developing a time and plan of action. 
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Level 4 

There is a clearly defined, ongoing self-review process in place, and all stakeholders (staff, parents/ 
community, students) can articulate their roles and responsibilities for student achievement. A recognition 
and intervention system is being implemented for all stakeholders that is directly related to achievement of 
the school's identified standards and learner outcomes, and the district goals and expectations. 



Recognition 
A recognition system is implemented and is 
consistently supported by all stakeholders. 
This system identifies and provides appropriate 
recognition to stakeholders who have contributed 
to improved student achievement and progress 
toward the school's goals and expectations. 



Intervention 
A system of intervention is implemented and 
consistently supported by all stakeholders. 
This system identifies and provides appropriate 
support for stakeholders requiring assistance 
to improve studwit achievement and meet school 
goals and expectations 



Level 3 

An ongoing self-review process has been developed and most stakeholders can articulate their roles and 
responsibilities for student achievement A recognition and intervention system is being implemented for 
most stakeholders, and it is directly related to achievement of the school's identified standards and learner 
outcomes, and the district goals and expectations. 



Recognition 
A recognition system is implemented to 
celebrate contributions stakeholders have made 
to improving student achievement and progressing 
toward the school's goals and expectations. 



Intervention 
A system of intervention is implemented which 
identifies and provides appropriate support for 
stakeholders requiring assistance to improve 
student achievement and meet school goals and 
expectations. 



Lm]_2 

The school is working on implementing a self-review process, and identifying the roles and 
responsibilities of stakeholders ixi improving student achievement A recognition and intervention system 
is being developed. Some recognition and intervention processes may be in place, but they are not 
necessarily related to achievement of the school's identified standards and learner outcomes. 



Recognition 
There is a process for recognition of stakeholders, 
and the recognition is sometimes related to student 
achievement and the school's goals and expectations. 



Intervention 

There are inten/ention processes in place at the 
schools which arp. sometimes related to the 
improvement of student achievement and 
completion of the school goals and expectations. 



The school has not yet developed a self-review process or identified the roles and resjponsibilities of all 
stakeholders in improving student achievement Some recognitioii and intervention processes are in place, 
but they arc not aligned with the school's identified standards and learner outcomes. 



RecoenitiQii 

There is recognition of stakeholders which is not 
necessarily related to student achievement and the 
school's goals and expectations. 



Intervention 

There are some intervention processes in place 
which are not focused on the improvement of 
student achievement and completion of the 
school's goals and expectations 
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Level 4 

Student achievement is the primary focus of the accountability reporting process, and 
accountability reporting is an integral part of school planning processes. There is a comprehensive 
understanding among all stakeholders of the school's goals, expectations, timelines, and 
responsibilities. There is a well-defined, two-way system of communication among stakeholders 
about student achievement and progress toward the school goals and expectations. The 
governance team coordinates an ongoing self-review process to evaluate and communicate school 
performance in accountability for student achievement. 



Level 3 

Student achievement is the focus of the accountability reporting process, and accountability 
reporting is part of school planning processes. There is an understanding among most 
stakeholders of the school's goals, expectations, timelines, and responsibilities. A two-way 
communication system involves most stakeholders in discussions about student achievement and 
progress toward the school goals and expectations. The governance team coordinates an ongoing 
self-review process to evaluate and communicate school performance in accountability for student 
achievement 



Level 2 

The school conducts some accountability reporting activities which may focus on student 
achievement; accountability reporting is not fully integrated into the school planning processes. 
There is a partial understanding, which varies among different stakeholder groups, of the school's 
goals, expectations, timelines, and responsibilities. While there is not yet a schoolvvide system of 
communication, some stakeholders (e.g., grade levels/departments, school committees, 
parent/teacher conferences) are involved in discussions about student achievement and progress 
toward school goals and expectations. Communication is generally one-way, from the school to 
the parents and/or community. The governance team is lieginning to develop a self-review process 
to evaluate and communicate school performance in accountability for student achievement. 



Level 1 

School reporting activities may or may not focus on student achievement, and accountability 
reporting is not part of the school planning processes. Stakeholder groups have a limited 
understanding of the school's goals, expectations, timelines, and responsibilities. Teachers may 
spend some time discussing student achievement and/or reporting to parents on an individual basis; 
there is noinimal involvement of or commimication with other stakeholders. The governance team 
is be^rjriing to discuss their role in conducting a school self-review to evaluate and communicate 
school perfonnance in accountability for student achievement 
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APPENDIX D 
ACCOUNTABILITY RUBRIC SUMMARY MATRIX 
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SELF-ASSESSMENT AND VALIDATION REVIEW RATING SHEET 
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